Skip to content

Add concurrent pull XComs example in Go-SDK e2e test#68934

Open
jason810496 wants to merge 3 commits into
apache:mainfrom
jason810496:ci/go-sdk/add-goroutine-example
Open

Add concurrent pull XComs example in Go-SDK e2e test#68934
jason810496 wants to merge 3 commits into
apache:mainfrom
jason810496:ci/go-sdk/add-goroutine-example

Conversation

@jason810496

@jason810496 jason810496 commented Jun 24, 2026

Copy link
Copy Markdown
Member

Why

One of the most significant feature of Golang programming language is the goroutine. We should definitely add an example to showcase the injected client is safe to access the Airflow models (XCom, Variable, Connection) within and across goroutines for Go-SDK.

What

Add a new Go Dag example that pull the XComs with WaitGroup which will also be tested in e2e test.


Was generative AI tooling used to co-author this PR?

@jason810496 jason810496 self-assigned this Jun 24, 2026
@jason810496 jason810496 force-pushed the ci/go-sdk/add-goroutine-example branch from b80f7ee to dface76 Compare June 24, 2026 09:42
@jason810496 jason810496 marked this pull request as ready for review June 25, 2026 07:31
Comment thread go-sdk/example/bundle/concurrentxcom/concurrentxcom.go Outdated
@jason810496 jason810496 force-pushed the ci/go-sdk/add-goroutine-example branch from dface76 to 341e260 Compare June 25, 2026 12:12

@uranusjr uranusjr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor suggestion (the current approach works with Go 1.24 so it’s not critical)

Comment on lines +86 to +92
for i, key := range keys {
wg.Add(1)
go func() {
defer wg.Done()
concResults[i], errs[i] = pull(key)
}()
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for i, key := range keys {
wg.Add(1)
go func() {
defer wg.Done()
concResults[i], errs[i] = pull(key)
}()
}
for i, key := range keys {
wg.Add(1)
go func(i int, key string) {
defer wg.Done()
concResults[i], errs[i] = pull(key)
}(i, key)
}

suggested by Claude to be explicit. I don’t know what Go people generally prefer (according to Claude, the syntax without arguments only works after 1.22) but passing explicit values to a lambda call is generally better in many languages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants